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Abstract 

Abstract: This paper provides an illustrative example of an approach to 
creating and reporting individual profiles of engagement in particular 
behaviours in an online asynchronous discussion (OAD). Individual results of 
analysis of transcripts of an OAD can provide insights different from those 
gained by focusing on aggregate measures of group behaviours. In this 
case, we focused on individual behaviours associated with Problem 
Formulation and Resolution (PFR) in a one-month long OAD with seven 
graduate students. The transcripts of each participant were analysed for 
patterns of PFR behaviours using a previously designed instrument. 
Individual profiles of the seven participants were created. The paper 
provides examples of how the approach facilitated identification and 
comparison of individual weaknesses and strengths. Also provided are 
examples of how individual profiles might be useful in professional 
development and instructional contexts for formative or summative 
assessment purposes. 

Resume: L'article presente un exemple i I lustre d'une methode consistant a 
creer et a declarer des profils d'engagement individuels envers des 
comportements particuliers dans le cadre d'une discussion electronique 
asynchrone (DEA). Chaque resultat des analyses de transcription d'une DEA 
peut donner lieu a un apergu different de celui obtenu en se concentrant sur 




des mesures globales de comportements en groupe. Dans le present cas, 
nous nous concentrons sur les comportements individuels associes a la 
formulation et a la resolution de problemes (FRP) dans le cadre d'une DEA 
qui s'est etendue sur une periode d'un mois a laquelle participaient sept 
diplomes. Nous avons analyse les transcriptions de chaque participant en 
cherchant des tendances de comportements de FRP au moyen d'un 
instrument identifie au prealable. Des profils individuels de chacun des sept 
participants ont ete crees. L'article i I lustre comment la methode facilite 
I'identification et la comparaison de chacune des forces et des faiblesses. II 
presente aussi comment chacun des profils peut s'averer utile dans le cadre 
du perfectionnement professionnel ainsi que des contextes d'instruction aux 
fins de revaluation formative ou sommative. 

Introduction 

Content analysis of online asynchronous discussions (OADs) first received attention in the 
pivotal work of Flenri (1992) who proposed a seminal model for the analysis of dimensions 
of the learning process. Since then, researchers have continued to use models and 
instruments that define processes or indicators of the particular construct under study to 
identify instances of particular behaviours in the transcripts of OADs. Some studies have 
analysed the construct of critical thinking (e.g., Newman, Johnson, Cochrane & Webb, 
1996). Gunawardena, Lowe and Anderson (1997) and Kanuka and Anderson (1998) used 
an interaction analysis model to analyse knowledge construction. Flara, Bonk and Angeli 
(2000) used Flenri's model to conduct content analysis of a discussion for five dimensions 
of the learning process. Garrison, Anderson and Archer (2000) developed a Community of 
Inquiry model with three components, cognitive presence, social presence, and teaching 
presence in order to provide a conceptual framework and tool for analysis of use of 
computer-mediated communication in educational contexts. Subsequently, Rourke, 
Anderson, Garrison and Archer (2001a) used this same tool to analyse and compare two 
transcripts for social presence. 

Results of analyses of transcripts of discussions are often presented as aggregate or group 
measures. This approach to reporting may be appropriate to assess whether the group as 
a whole did or did not engage in behaviours related to a particular construct (e.g., 
knowledge construction, critical thinking, etc.). These aggregate results can then be used 
to improve the design and effectiveness of the overall discussion. In other cases, the 
approach may be used to refine analysis techniques, to test models or to identify the 
groups' levels of engagement in a particular construct. As an illustration of the latter 
Gunawardena et al. (1997) analysed an online debate to determine if the group of 554 
discussants in 35 countries engaged in knowledge construction. The authors note "the 
objective was to evaluate the learning process taking place among the group of 
participants rather than to assess individual student performance" (p. 405). The approach 
to reporting aggregate results may also help assess whether the discussion successfully 
engaged participants in the targeted behaviours. For example, when Gunawardena et al. 



applied their model to the analysis of the discussion they found that participants engaged 
primarily in just one phase of knowledge construction. The approach of presenting 
aggregate results can also be used to compare and contrast different discussion groups or 
different groups of students in the same discussion. For example, Newman et al. (1996) 
compared critical thinking in face-to-face seminars with computer conference discussions. 
Kim and Bonk (2002) used Curtis and Lawson's (2001) model to assess collaboration by 
cross-cultural groups of students in an online discussion. 

Content analysis of OADs using models and instruments can also support identification and 
means of engagement in targeted behaviours by individuals and not only groups. Different 
insights may be gained from using one approach or the other. For example, analysis at the 
individual level can provide insight into a discussant's weaknesses or strengths and help 
identify areas needing attention. Individual results may also be useful in contexts of 
professional development where the focus is on developing skills related to constructs 
such as collaboration or problem-solving. At the same time, the approach could be useful 
for assessment and evaluation purposes. 

There are some examples in the literature of this approach to creating and reporting 
individual as opposed to aggregate results of analysis of behaviours in OADs. Bullen (1997) 
reported individual measures of participation and critical thinking in an online discussion 
conducted in a university course. Using these measures together with information from 
individual participants such as age, gender, motivation, and educational level, Bullen found 
"apparent relationships between participation levels, critical thinking levels and student 
characteristics" (p. 151). In de Laat and Lally's (2003) study, results of computer-assisted 
analysis of collaborative learning and tutoring processes in an OAD among professionals 
were presented both for the group and for individuals. Individual results helped identify 
learning patterns and a variety of individual roles in tutoring. Perkins and Murphy (2006) 
analysed the transcripts of an OAD using a critical thinking model and presented results 
with a focus on individual engagement in critical thinking behaviours. This approach served 
to highlight similarities and differences between participants, revealing substantial 
differences in the proportions of engagement in critical thinking processes among different 
students. 

While these examples show evidence of interest in reporting individual results, the more 
common tendency is to report aggregate results. For this reason, there are fewer cases 
where content analysis of OADs has focused on compiling results of individual engagement 
in targeted behaviours. The purpose of this paper, therefore, is to offer an illustrative 
example of an approach to reporting individual profiles of engagement in an OAD. In this 
example, we focused on a month-long discussion with a group of seven graduate students 
engaged in a discussion designed to promote Problem Formulation and Resolution (PFR). 

The paper begins with a description of the method used for analysing the discussion 
transcripts using a model for identifying and measuring PFR in an OAD. The individual 
profiles of engagement in PFR are presented in the following section. A summary of the 



profiles is then presented in table format for purposes of comparisons between individuals. 
The discussion section focuses on illustrating the types of insights that might be gleaned 
from this approach to profiling individual behaviours. 

Methods 

Participants were seven graduate students enrolled in a Counselling Psychology course in 
the fall of 2004 who volunteered to complete a one-month long online discussion designed 
for engagement in Problem Formulation and Resolution (PFR). The problem or issue 
presented for discussion was that of promoting parental involvement in schools. The 
unmoderated discussion consisted of eight discussion prompts or tasks such as the 
following: "You have had the opportunity to read one research article on the problem. 
Compose and post a message in which you describe how your understanding of the 
problem has changed as a result of having read the article" (Murphy, 2004b). The first five 
tasks were designed to support engagement in Problem Formulation while the remaining 
three tasks were designed to support Problem Resolution. Therefore there was greater 
emphasis on Formulation (63% of tasks) than on Resolution (37% of tasks). In other 
words, the discussion was designed to engage participants in formulating and 
understanding the problem before attempting to solve it. 

Once all eight tasks were completed, participants' transcripts were compiled, printed, and 
then coded for PFR behaviours using the instrument outlined in Table 1. This was a second 
iteration of the instrument designed for identification and measurement of PFR (see 
Murphy, 2004a). The instrument is divided into two main categories: Problem Formulation 
and Problem Resolution. Two processes are associated with Problem Formulation: Defining 
the problem space and Building knowledge. Three are associated with Problem 
Resolution -.Identifying solutions-, Evaluating solutions-, and Acting on solutions. Finally, 
there are a total of 19 indicators associated with these five processes. Table 1 presents 
the instrument with the codes used for analysing the transcripts. 



Table 1. Instrument for Identifying and Measuring Problem Formulation arid 
Resolution in an Online Asynchronous Discussion (Murphy, 2004a) 


Process 

Indicator 

Codes 

FORMULATION 


Defining 

problem 

space 

• Agreeing with problem as presented in OAD 

• Specifying ways in which problem may manifest itself 

• Redefining problem w ithin problem space 

• Minimizing and or denying the problem 

• Identifying extent of problem 

• Identifying c auses o f problem 

• Articulating problem outside problem space 

FDA 

FDS 

FDR 

FDM 

FDIe 

FDIc 

FDAp 

Building 

knowledge 

• I dentifying unknowns in knowle dge 

• Accessing and reporting on sources of information 

• Identifying value of information 

• Re fie cting on one' s thinking 

FBI 

FBA 

FBIv 

FBR 

RESOLUTION 


Identifying 

solutions 

• Proposing solutions 

• Hypothesizing about solutions 

RIP 

RIH 

Evaluating 

solutions 

• Agreeing with solutions proposed by others 

• Weighing and comparing alternative solutions 

• Critiquing solutions 

• Rej e cting eliminating solut ions j udge d unwork able 

REA 

REW 

REC 

RER 

Acting on 
solutions 

• Planning to act 

• Reaching conclusions, or arriving at an understanding of 
problem 

RAP 

RAR 


The unit of analysis chosen for coding was the paragraph. A total of 260 paragraphs were 
coded. Unlike the thematic unit or unit of meaning, syntactic units such as the paragraph 
have the advantage of being objectively identified and constant within a transcript (Fahy, 
2001; Rourke, Anderson, Garrison, & Archer, 2001b). The transcripts were first coded 
simultaneously by two separate coders (coder A and coder B). Inter-rater reliability was 
calculated using Cohen's Kappa. The value was 0.591. A subsequent second stage of 
coding was conducted by the two coders in conjunction with the principal investigator 
(coder C) and creator of the instrument. All discrepancies were discussed until 100% 
agreement was reached among the three coders, resulting in a consensual coding 
(A+B+C). It is the results of this second stage of coding that were used to create the 
profiles presented in this paper. 

Individual profiles 

This section presents the individual profiles of engagement in PFR based on analysis of the 
transcripts using the instrument. Pseudonyms are used for participants. For each 
individual, a figure first summarises the percentages of units coded for the behaviours 
associated with the various indicators for Problem Formulation and Resolution. The figure is 
followed by the descriptive profiles of the individual, presented in two parts, first for 













Problem Formulation and, second, for Problem Resolution. 

ft9fHSi3fibn ura 

In spite of the fact that the discussion was designed to engage participants in as much 
Formulation as Resolution, Laura's engagement in Formulation counted for only 36% of all 
units coded for in her transcript. 12% of the units coded for Formulation involved 
identifying causes while another 12% involved reflecting on her thinking. Finally, 3% of 
units accounted for each of four other indicators: agreeing with the problem as presented; 
specifying ways the problem manifests itself; accessing and reporting on sources of 
information; and identifying the value of knowledge. Of the 11 indicators associated with 
Formulation in the coding instrument, only six of these were actually coded for in her 
transcript. In this regard, Laura did not redefine, minimise, or identify the extent of the 
problem, nor did she did articulate a problem outside the problem space or identify 
unknowns in knowledge. 



Figure 1 Summary of Laura's engagement in Formulation and Resolution. 

Resolution 

Compared with other participants, Laura exhibited the highest percentages for engagement 
in Resolution (64% of units in her transcript). Only three of the total eight indicators for 
Resolution were privileged. Seventeen percent of the units were coded as hypothesising 
about solutions. Another 17% were coded as agreeing with solutions proposed by others, 
and 15% were coded as proposing solutions. Reaching conclusions or arriving at an 
understanding of the problem involved 9% of coded units in Laura's transcript, which was 
the highest percentage for this behaviour as compared with other participants. Three 
percent of units each were coded for critiquing solutions and planning to act. Laura did not 
weigh and compare solutions or reject solutions judged unworkable. Laura's overemphasis 
on solutions is highlighted by the fact that her very first posting in the discussion was 















coded as hypothesising about solutions, even if she had not yet formulated the problem. 
While four of the seven participants did not hypothesise about solutions, Laura devoted 
17% of units to this behaviour. Laura was the only participant who did not only emphasise 
proposing solutions but also privileged hypothesising about solutions when engaging in 
Resolution. 

fB?fHSlSfiih omas 

The percentage of units coded in Thomas' transcript for engagement in each of the 
categories of Formulation and Resolution was 50%. Within Formulation, Frank engaged in 
eight of the 11 possible types of behaviours. In terms of the units coded in his transcript 
for those behaviours, accessing and reporting on sources of information was the behaviour 
coded for most frequently (14% of units). Compared with other participants, Thomas' 
transcript together with Frank's exhibited the highest percentages for agreement with the 
problem as presented (6%). However, Thomas did not redefine or minimise the problem 
nor did he identify unknowns in knowledge. 



°-o e: rar.i ay 


Figure 2. Summary of Thomas' engagement in Formulation and Resolution. 

Resolution 

Twenty-three percent of units in Thomas' transcript corresponded to proposing solutions, 
which is the behaviour he favoured over the other four Resolution behaviours coded for in 
his transcript. The other behaviour he favoured within Resolution was agreeing with 
solutions proposed by others (9% of units). He also engaged in planning to act and 
reaching conclusions (6% each), and in critiquing solutions and rejecting solutions judged 
unworkable (3% each). Of interest is the fact that Thomas' transcript exhibited the highest 
percentage among all participants for engagement in planning to act. Also of note is the 
fact that only Thomas' transcript reflected engagement in rejecting solutions judged 
unworkable. Thomas did not engage in hypothesising about solutions, nor did he weigh and 

















compare alternative solutions. 


PS?MSi$Mfl drea 

In marked contrast with the other participants, Andrea's engagement in Formulation was 
almost double her engagement in Resolution. For example, 65% and 35% of the units in 
Andrea's transcript accounted for Formulation and Resolution respectively, whereas in 
Laura's transcript, the percentages were almost the reverse. Andrea engaged in nine of the 
11 Formulation behaviours identified in the instrument. Her focus on Formulation is 
illustrated by the fact that her engagement in one particular behaviour within Formulation, 
identifying the value of knowledge, accounted for 22% of her units. This percentage was 
double that of other participants' engagement. She also favoured accessing and reporting 
on sources of information (13% of units). Andrea tended to engage in articulating a 
problem outside the problem space or digressing, for example talking about her work 
situation. Nine percent of her units were coded as articulating a problem outside the 
problem space, which was the highest percentage for this behaviour among all 
participants. Of interest is the fact that only two participants, Andrea and Susan, engaged 
in minimising the problem. Andrea also engaged in agreeing with the problem as 
presented, specifying ways the problem manifests itself, redefining the problem, identifying 
its causes, and reflecting on her thinking. She did not engage in identifying the extent of 
the problem or in identifying unknowns in knowledge. 



Figure 3. Summary of Andrea's engagement in Formulation and Resolution. 

Resolution 

Like Thomas, Andrea did not engage frequently in Resolution until half-way through the 
discussion. Within Resolution, she privileged proposing solutions. In fact, she engaged 
more in proposing solutions than in any other behaviour related to Formulation or 
Resolution. Proposing solutions accounted for 22% of units in her transcript. In contrast, 
percentages were low for the other four Resolution behaviours Andrea engaged in, each of 



















which accounted for 5% or less of units coded. Andrea's focus on proposing solutions 
above any other Resolution behaviour is illustrated by the fact that the percentage of units 
coded in her transcript for agreeing with solutions proposed by others (5%) was the lowest 
as compared with the other participants. The other behaviours related to Resolution that 
Andrea engaged in were critiquing solutions, planning to act, and arriving at a conclusion. 
She did not engage in hypothesising about solutions, weighing solutions, or rejecting 
solutions judged unworkable. 

&>9fHBi8(iSa ro1 

Carol concentrated both on Formulation and on Resolution (47% and 53% of her units 
respectively). Compared with other participants, within Formulation, Carol exhibited the 
highest percentage of units coded as specifying ways the problem manifests itself (6% of 
units). Carol privileged accessing and reporting on sources of information and identifying 
the value of information (13% of units each). In her transcript, only one unit was coded as 
identifying causes of the problem, which accounted for 3% of her units. Of interest within 
Formulation is the fact that only Carol, Susan and Andrea engaged in redefining the 
problem within the problem space. Within Formulation, Carol also agreed with the problem 
as presented, and reflected on her thinking. Flowever, she did not minimise the problem, 
identify the extent of the problem, articulate a problem outside the problem space, or 
identify unknowns in knowledge. 

Resolution 

Within Resolution, Carol concentrated on proposing solutions and on agreeing with 
solutions. Carol's and Erin's transcripts presented the highest percentages of units coded 
as proposing solutions (25%). She also concentrated on hypothesising about solutions, 
critiquing solutions, and reaching conclusions. Fler engagement in hypothesising about 
solutions accounted for 6% of units in her transcript. Of interest is the fact that this 
represented the second highest percentage in the discussion for that behaviour, whereas 
four other participants did not engage in hypothesising about solutions. Carol did not 
engage in other behaviours associated with Resolution, such as weighing and comparing 
solutions, rejecting solutions judged unworkable, or planning to act. 




Figure 4. Summary of Carol’s engagement in Formulation and Resolution. 
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Fifty-four percent and 46% of the units in Frank's transcript were coded for Formulation 
and Resolution behaviours respectively. Frank's and Thomas' engagement in accessing and 
reporting on sources of information accounted for 14% of their transcripts, which was the 
highest percentage coded for this behaviour among all participants. This is precisely the 
Formulation behaviour Frank engaged in most, followed by identifying the value of 
information (10% of units). Fie identified causes of the problem (9%), reflected on his 
thinking (9%), and agreed with the problem as presented (6%). Finally, he also specified 
ways the problem manifests itself and identified the extent of the problem (3% of units 
each). Frank did not engage in redefining the problem within the problem space, 
minimising the problem, articulating a problem outside the problem space, or identifying 
unknowns in knowledge. 

Resolution 

Like Laura, Frank engaged in Resolution from early on in the discussion. Within Resolution, 
most units coded in Frank's transcript corresponded to proposing solutions and agreeing 
with solutions proposed by others (17% and 23% respectively). His engagement in 
agreeing with solutions proposed by others was the highest among all participants. 
Whereas Frank concentrated on proposing solutions and agreeing with solutions, he only 
minimally hypothesised about solutions and critiqued solutions. Like most other 
participants, Frank did not weigh and compare alternative solutions or reject solutions. 
With regard to Resolution, it is also of note that Frank was the only participant who did not 
engage in reaching conclusions. Like other three participants (Carol, Susan, and Erin), he 
did not engage in planning to act. 

















Figure 5. Summary of Frank's engagement in Formulation and Resolution. 
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Fifty-six percent of units coded in Susan's transcript related to Formulation behaviours. 
Susan engaged in eight of the 11 Formulation behaviours identified in the instrument. 
Compared with other participants, Susan exhibited the highest percentage of units coded 
as identifying the extent of the problem (9%), minimising the problem (6%), and 
redefining the problem (4%). Within Formulation, only one other participant engaged in 
minimising the problem and only two others in redefining the problem. She also identified 

causes of the problem, articulated a problem outside the problem space, accessed and 

reported on sources of information, identified the value of information, and reflected on her 
thinking. Flowever, Susan and Erin were the only participants who did not agree with the 
problem or specify ways the problem manifests itself. 

Resolution 

Within Resolution, Susan privileged two behaviours, proposing solutions and agreeing with 
solutions proposed by others. Nineteen percent of units in her transcript were coded for 
each of these behaviours. Compared with the other participants, Susan's transcript 
exhibited the second highest percentage of units coded as agreeing with solutions 

proposed by others. She also engaged in critiquing solutions (3%) and reaching 

conclusions (3%). Flowever, she did not engage in hypothesising about solutions, like 
Thomas, Andrea, and Erin, and she did not weigh and compare alternative solutions, reject 
solutions, or plan to act. 


















Figure 6 Summary of Susan's engagement in Formulation and Resolution. 

Profile of Erin 

Erin concentrated both on Formulation and on Resolution (53% and 47% of units coded 
respectively). She engaged in five of the 10 Formulation behaviours present in the 
instrument. Erin's transcript presented the highest percentage of units coded for identifying 
causes of the problem (20%). She also focused on accessing and reporting on sources of 
information, identifying the value of information, articulating a problem outside the 
problem space, and reflecting on her thinking. Compared with other participants, her 
engagement in reflecting on her thinking exhibited the lowest percentage of units coded 
(5%). Erin and Susan were the only participants who did not engage in agreeing with the 
problem as presented and in specifying ways the problem manifests itself. Erin neither 
engaged in redefining the problem, minimising it, indicating its extent, nor recognising 
unknowns in knowledge. 
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Figure 7. Summary of Erin’s engagement in Formulation and Resolution. 

Sussh/sOh" 

Erin engaged in four of the eight Resolution behaviours identified in the instrument. She 
focused primarily on proposing solutions and agreeing with solutions. Twenty-five percent 
of the units in Erin's transcript were coded as proposing solutions. Together with Carol, Erin 
exhibited the highest percentage for engagement in this behaviour compared with other 
participants. In comparison with the other participants' transcripts, Erin's transcript also 
exhibited the highest percentage of units coded as critiquing or questioning solutions (5%). 
She also engaged in reaching conclusions (2% of units). Like other participants, she did 
not engage in hypothesising about solutions, weighing and comparing solutions, rejecting 
solutions judged unworkable, or planning to act. 

Summary of profiles 

To support comparison among participants, Table 2 presents individual summaries of their 
engagement in Problem Formulation and Resolution. Results are provided for each 
participant in terms of the percentage of units coded in relation to their transcript as a 
whole. For each participant, the percentage of units coded for the 11 indicators of Problem 
Formulation and the eight indicators of Problem Resolution in the instrument are 
presented. Total percentages for engagement in the categories of Formulation and 
Resolution are also provided. For a full description of the indicators, see Table 1. 




















Table 2. Summaries of Participants’ Engagement in Problem Formulation and 
Problem Resolution by bidicator arid by Category 




Laura 

Thomas 

Andrea 

Carol 

Frank 

Susan 

Erin 


FDA 

•y 

3 

6 

*> 

X 

•y 

6 

0 

0 


FDS 


•y 

2 

6 

•y 

0 

0 


FDR 

0 

0 

7 

x 

3 

0 

4 

0 


FDM 

0 

0 

7 

x 

0 

0 

6 

0 


FDIe 

0 

•y 

3 

0 

0 

•y 

9 

0 

Formulation 

FDIc 

12 

6 

5 

3 

9 

9 

20 


FDAp 

0 

6 

9 

0 

0 

4 

5 


FBI 

0 

0 

0 

0 

0 

0 

0 


FBA 

3 

14 

13 

13 

14 

9 

13 


FBIv 

3 

3 

77 

XX 

13 

10 

9 

10 


FBR 

12 

S 

s 

6 

9 

6 

5 

Total % 


36 

50 

65 

47 

54 

56 

53 



Laura 

Thomas 

Andrea 

Carol 

Frank 

Susan 

Erin 


RIP 

15 

23 

22 

25 

17 

19 

25 


RIH 

17 

0 

0 

6 

•y 

0 

0 


REA 

17 

9 

5 

16 

23 

19 

15 

Resolution 

REW 

0 

0 

0 

0 

0 

0 

0 

REC 

•y 

3 

•y 

3 

4 

•y 

3 

•y 

3 

5 


RER 

0 

3 

0 

0 

0 

0 

0 


RAP 

•y 

6 

7 

X 

0 

0 

0 

0 


RAR 

9 

6 

7 

X 

•y 

3 

0 

•y 

3 

7 

X 

Total % 


64 

50 

35 

53 

46 

44 

47 


Discussion 

This paper provides an illustrative example of an approach to creating and reporting 
individual profiles of engagement in particular behaviours in an OAD. In this case, we 
focused on individual behaviours associated with Problem Formulation and Resolution 
(PFR) in a one-month long OAD with seven graduate students. The individual profiles 
represent examples of how this approach can facilitate the identification of differences in 
individual participants' patterns of engagement in Problem Formulation and Problem 
Resolution. The approach can also facilitate comparison and contrasting of those patterns. 
For example, in terms of individual participants' engagement in Formulation and Resolution 






































in the discussion, Andrea favoured Formulation (65% of units in her transcript); in 
contrast, Laura favoured engagement in behaviours related to Resolution (64% of units); 
while the other five participants engaged almost equally in both (see Table 2). These types 
of results can help identify weaknesses and strengths in participants' abilities to engage in 
Problem Formulation and Resolution. The results could be used for assessment purposes or 
for guiding participants in a context of learning. 

The individual profiles could also be useful in relation to the design and goals of the 
discussion. In our case, the discussion was designed to place greater emphasis on 
Formulation (63% of tasks) than on Resolution (37% of tasks). Andrea's results of 
engagement in Formulation and Resolution behaviours, with 65% and 35% of units coded 
respectively, matched almost exactly the distribution of tasks in the discussion. As the 
only individual who clearly favoured Formulation over Resolution, she presents an example 
of a model of engagement in PFR. The results of analysis of her transcript illustrate how 
the approach might be useful in terms of identifying best practices. These best practices 
could subsequently be used to provide instructors and students with models of 
engagement in the behaviours for which the discussion was designed. 

In contrast to Andrea, Laura favoured Resolution. Laura's engagement, with 36% and 64% 
of her units corresponding to Formulation and Resolution respectively, was almost exactly 
the opposite of Andrea's. In this case, the focus on individual participants' behaviours, 
therefore, highlighted a weakness in Laura's engagement in relation to the behaviours the 
discussion targeted. The identification of such weaknesses might be useful for formative or 
summative evaluation purposes in an instructional context. In addition, this type of 
information might be useful in a context of professional development using OADs. If 
analysis were performed during the discussion, it might be possible to orient participants' 
behaviours so that they correspond more with the required discussion tasks. Laura's 
preference for Resolution instead of Formulation suggests that the design of the discussion 
was not always effective in promoting engagement in the targeted behaviours. These 
types of results can reveal aspects of the design that may need to be reconceptualised to 
help participants such as Laura engage more in Formulation. 

In relation to the indicators of Formulation and Resolution, the individual profiles illustrated 
how the approach can reveal the range of variation among participants' behaviours. If 
results for the whole group had been reported, it would not have been possible to compare 
participants' behaviours. In addition, aggregate results, especially when presented in 
terms of averages, may be misleading if there is a wide range between participants. 
Aggregate results may also fail to reveal the whole range of behaviours each individual 
engaged in. In our OAD, for some PFR behaviours, individual results revealed important 
differences between participants. It was possible to identify which participants engaged 
more or less than others in specific behaviours. For example, within Formulation, 
engagement in identifying the value of information ranged between 3% and 22% of units 
among participants. Andrea's engagement in this behaviour, with 22% of units in her 
transcript coded for it, surpassed the other discussants' engagement. Within Resolution, 



the largest range among participants in terms of engagement in one particular behaviour 
corresponded to agreeing with solutions proposed by others. Only 5% of units in Andrea's 
transcript related to this behaviour, whereas Frank devoted 23% of units to it. In contrast 
with the other participants, who favoured proposing solutions within Resolution, Laura was 
the only one who devoted more units in her transcript to hypothesising about solutions 
(17%) than to proposing solutions (15%). 

Finally, the focus on behaviours associated with the indicators at the individual level 
helped identify when individual participants only engaged in a limited number of behaviours 
rather than engaging in a variety of behaviours. If, for example, we considered 
engagement in as many PFR behaviours as a desired outcome of the discussion, then Erin 
would show a weakness in terms of her engagement in Formulation. Erin focused mostly on 
one particular behaviour, identifying causes of the problem, while she did not engage in a 
variety of other Formulation behaviours. Within Resolution, most participants favoured 
proposing solutions and agreeing with solutions proposed by others, while nobody engaged 
in weighing and comparing alternative solutions. For example, almost all of Andrea's 
engagement in Resolution corresponded to just one behaviour, proposing solutions. 

Conclusion 

This paper illustrated the value of an approach to creating and reporting individual profiles 
of discussants' behaviours in an OAD. This approach supported comparing and highlighting 
the particular behaviours participants tended to engage in over other behaviours. It 
supported identification of the behaviours individuals favoured and identification of the 
behaviours in which they engaged only minimally. Comparing and contrasting individual 
participants' engagement was useful to gain insight into their weaknesses and strengths. 
The approach also revealed patterns of engagement in PFR behaviours in the discussion 
and the range of behaviours in which participants engaged. It also indicated whether 
individual patterns of behaviour matched or did not match the purpose and design of the 
discussion which was intended to engage participants more in Formulation than in 
Resolution. The results could be used for various purposes, such as to assess the overall 
effectiveness of a discussion and subsequently redesign it, to guide and support individuals 
in professional development contexts, and to support formative and summative 
assessment of individuals. 

In spite of its usefulness, the approach presents some limitations. These relate to the 
resources needed to compile individual profiles. If used for individual assessment, the 
approach would be onerous for the instructor in terms of the time needed to code 
transcripts and analyse results. In this case, computer-assisted analysis might be helpful. 
Alternatively, students might be required to self-analyse and evaluate their contributions 
or other students' contributions. Another limitation of the approach is that it does not 
provide insight into why participants did or did not engage in particular behaviours or why 
they engaged in some behaviours over others. In this regard, the approach could be 
enhanced by combining analysis of individual transcripts with further qualitative data that 
provide insight into individuals' thinking. Hara et al. (2000) suggested that content 



analysis of online discussions should be complemented by additional tools such as 
interviews and retrospective analysis to gain additional insight. Some studies have used 
analysis of online discussion transcripts in combination with other data collection methods 
such as surveys, individual interviews, focus groups, or recalls (e.g., Bullen, 1997; Cheung 
& Hew, 2004; de Laat & Lally, 2003; Rourke & Anderson, 2002). 

The study is limited in its small number of participants as well as its use of only one 
instrument. However, it does provide an indication of alternative approaches that can be 
taken in the context of content analysis of OADs. Further studies might make use of other 
instruments and compare their value. Researchers may also consider behaviours 
associated, not only with Problem Formulation and Resolution, but also with critical 
thinking, collaboration, knowledge construction, social presence, or any combination of 
these. They may also wish to analyse the transcripts of discussions in subject areas other 
than Counselling Psychology. Other studies might provide an opportunity to not only 
include graduate students, as was the case in this study, but also undergraduate students. 
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